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Abstract 

Malware is widely acknowledged as a growing threat with hundreds of thousands of new samples 
reported each week. Analysis of these malware samples has to deal with this significant quantity but 
also with the defensive capabilities built into malware; Malware authors use a range of evasion 
techniques to harden their creations against accurate analysis. The evasion techniques aim to disrupt 
attempts of disassembly, debugging or analyse in a virtualized environment. 

This talk catalogs the common evasion techniques malware authors employ, applying over 50 different 
static detections, combined with a few dynamic ones for completeness. We validate our catalog by 
running these detections against a database of 4 million samples (the system is constantly running and 
the numbers will be updated for the presentation), enabling us to present an analysis on the real state of 
evasion techniques in use by malware today. The resulting data will help security companies and 
researchers around the world to focus their attention on making their tools and processes more efficient 
to rapidly avoid the malware authors' countermeasures. 

This first of its kind, comprehensive catalog of countermeasures was compiled by the paper's authors 
by researching each of the known techniques employed by malware, and in the process new detections 
were proposed and developed. The underlying malware sample database has an open architecture that 
allows researchers not only to see the results of the analysis, but also to develop and plug-in new 
analysis capabilities. The system will be made available in beta at Black Hat, with the purpose of 
serving as a basis for innovative community research. 


1. Introduction basement for decisions, but still they are the 

majority of the cases. 

Besides the common sentences among researchers This works analyzed millions of malwares focusing 

and industry regarding the amount of new samples in their protection mechanisms. We divided the 

every day (near to the hundred thousand daily), still protection mechanisms in 4 different categories: 

the analysis efforts focus on automating a specific • Anti-Debugging: Techniques to 

task or automate the analysis of only one sample. compromise debuggers and/or the 

Researchers around the globe have many debugging process 

challenges to contribute in combating new • Anti-Disassembly: Techniques to 

malware, since they either lacks the access to the compromise disassemblers and/or the 

samples or access to the computing power to disassembling process 

process them (or both). This limits the amount of • Obfuscation: Techniques to make the 

contributions coming from the academia and from signatures creation more difficult and the 

individual contributors. disassembled code harder to be analyzed by 

The situation created an industry full of incomplete a professional 

results and opinions. Analysis comprising just a . Anti-VM: Techniques to detect and/or 

few thousands of malware samples are not a 



compromise virtual machines 

Techniques that are not being currently being 
detected in the malware samples are also explained: 
we are constantly updating the system. 

The paper is organized as follows. Section 1.1 
discusses our methodology, the automated analysis 
system and some other choices made for this 
research. Section 2 provides the results of our 
analysis, while the rest of the paper discusses the 
technical details of the implementations 
themselves. Section 3 enumerates and details each 
of the anti-debugging techniques. Section 4 
discusses disassembly concepts and anti¬ 
disassembly and obfuscation techniques. Section 5 
discusses anti-VM techniques. Section 6 illustrates 
new techniques and advancements proposed by this 
work. Section 7 comprises the downloading links 
for getting updated versions of this paper and for 
downloading the developed examples to validate 
each of the detection anti-reverse engineering 
mechanisms. Section 8 concludes and provides 
future directions. Section 9 has some 
acknowledges. Finally, in Section 10, the 
references used in this work. 


Windows VM; 

• Static plugins: plugins that run outside of 
the VM 

It was developed a plugin that is a framework for 
disassembly-related analysis: 

• Facilitates the development of disassembly 
analysis code; 

• Speeds up the disassembly process for 
plugins; 

• Calls-back the plugins for specific 
instruction types; 

• Disassembly once, analyze all; 

• Care must be taken to detect disassembly 
attacks themselves. 

For this work, we disassembled and analyzed only 
PE sections explicitly marked as executable or 
where the entry point is located. 

The anti-reverse engineering techniques were 
detected in the malware samples through plugins. 
Before its deploy, each plugin was tested against 
883 PE files looking for bugs and for the quality of 
the detection coverage itself. 


1.1. Methodology 


2. Executive Summary 


The analysis performed in this work relied in a total 
of 72 cores and a 100GB of RAM distributed in 9 
different machines. 

We analyzed a bit more than 4 million samples 
(4,030,945). Packed samples were not analyzed 
individually: all packed samples using the same 
packer have been considered as one single unique 
sample. 

All our samples were 3.9MB or less in size 
(performance reasons). The only exception was the 
Flame malware due to its importance. 

We used mostly static techniques, but included a 
few dynamic ones for completeness: some 
techniques cannot be detect using only a static 
approach. 

The automated malware analysis system, called 
Dissect || PE, relies in plugins. Each application 
that reads a malware and produces an output is 
considered a plugin. There are: 

• Dynamic plugins: plugins that run inside a 


For this research, we analyzed 4.030.945 malware 
samples in our lab. As depicted in Chart 1, 34,79% 
were packed, and the top packer families are shown 
in Chart 2. 





Looking for anti-reverse engineering techniques in 
the top packer families, we had different results for 
the same packer family because of different 
versions. Being so, we detailed the techniques 
found in each version in Table 1. 
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Chart 4 - Packer Families of Malware Samples 
Targeting Brazilian Banks 


From this point on, and according to the proposed 
methodology in which each packer was analyzed 
once, the following numbers are related to the not 
packed samples. Additionally, in the next statistics, 
malware analysis algorithms that produce 
evidences were not considered. 

Chart 5 shows that 88,96% of the samples had at 
least one anti-reverse engineering technique 
detected. 


Table 1 - Packers Anti-Reverse Engineering 

The top packer families for malware samples 
targeting Brazilian banks were also analyzed. As 


shown in Chart 3, we found that 50,49% were 
packed, and the top packer families are depicted in 
Chart 4. 

Packed 

Not packed 

50,49% 

49,51% 



Chart 5 - Samples with Anti-Reverse Engineering 

As shown in Chart 6, 6,42% of the analyzed 
samples have implemented at least one protection 
mechanism in each of the four categories (named as 
fully armored samples in this work). 


Chart 3 - Packer Statistics of Samples Targeting 
Brazilian Banks 

















Chart 7 - Anti-Reverse Engineering Categories ctei 10 - Obfuscation Techniques 


So, anti-VM and obfuscation categories are 
considerably more prevalent in the samples with, 
respectively, 81,40% and 68,95%. 

The considered anti-debugging techniques in all of 
the statistics in this work relied on the techniques 
depicted in Chart 8. Additionally, the percentage of 
each considered anti-debugging technique 
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3. Anti-Debugging Techniques Adopted Static Detection: 


Some anti-debugging techniques are described in 
the next sections. 

Techniques currently covered by detection plugins 
will have an additional information: the algorithm 
used to detect such a technique. 

3.1. PEB NtGlobalFlag 

NtGlobalFlag is a field of PEB at offset 0x68 [1], 
The presence of such values is not a reliable 
debugger detection technique, but can be 
considered as an evidence: 
FLG_HEAP_ENABLE_TAIL_CHECK (0x10), 
FLG_HEAP_ENABLE_FREE_CHECK (0x20) 
and FLG_HEAP_VALIDATE_PARAMETERS 


( 1 ) 

IsDebuggerPresent is looked for in IAT. If found, 
the technique is considered as detected. 

( 2 ) 

A MOV instruction (mov, movsx, movzx) copying 
PEB address (fs:[0x30]) somewhere (X) is looked 
for and X is saved for future use: 

mov/movsx/movzx X,fs:[0x30] 


Then, later in the same function, another MOV 
instruction (mov, movsx, movzx) referencing the 


(0x40). This might be used to detect the presence ofBeingDebugged field ([X+0x2] in some of the 
a debugger. operands) is looked for: 


Adopted Static Detection: 

A MOV instruction (mov, movsx, movzx) copying 
PEB address (fs:[0x30]) somewhere (X) is looked 
for and X is saved for future use: 

mov/movsx/movzx X,fs:[0x30] 


mov/movsx/movzx opl,op2 -► where “[X+0x2]” is 
a substring of opl or op2 

RET was used to consider the end of a function. 

If this scenario happens, this anti-debugging 
technique is considered as detected. 


Then, later in the same function, a CMP (cmp, 
cmpxchg) or a MOV (mov, movsx, movzx) 
instruction referencing the NtGlobalFlag 
([X+0x68] in some of the operands) is looked for: 

cmp/cmpxchg/mov/movsx/movzx opl,op2 -» 
where [X+0x68] is a substring of opl or op2 

RET was used to consider the end of a function. 


3.3. CheckRemoteDebuggerPresent 

CheckRemoteDebuggerPresent() is a kernel32 
function that sets Oxffffffff in pbDebuggerPresent 
parameter if a debugger is present [1]. Internally, it 
uses NtQuerylnformationProcessO with 
ProcessDebugPort as a ProcessInformationClass 
parameter. This function can be used to detect the 
presence of a debugger. 


If this scenario happens, this anti-debugging Adopted Static Detection: 
technique is considered as detected. 

( 1 ) 


3.2. IsDebuggerPresent 


CheckRemoteDebuggerPresent is looked for in 

IsDebuggerPresent() is a kernel32 function that IAT. If found, the technique is considered as 
returns TRUE if a debugger is present [1]. detected. 

Internally, it uses PEB's BeingDebugged Field. 

Such approaches can be used to detect the presence (2) 
of a debugger. 


NtQuerylnformationProcess is looked for in IAT. If 



found, the technique is considered as an evidence ([X+0xl8] in some of the operands) is looked for: 
detected. 

? operands -► where “[X+0xl8]” is a substring of 
3.4. Heap flags any of the operands 


Process default heap (retrieved through 
GetProcessHeap() or PEB) has the following two 
fields of interest that are influenced by PEB- 
>NtGlobalFlags: Flags, at offset 0x0c in the heap, 
and ForceFlags at offset 0x10 in the heap [1]. The 
following values for each of the fields are not a 
reliable approach to detect a debugger, but can be 
considered as an evidence: 


RET was used to consider the end of a function. 

If this scenario happens, this anti-debugging 
technique is considered as detected. 

3.5. NtQuerylnformationProcess - 
ProcessDebugPort 


Flags: HEAP_GROWABLE (2), 

HEAP_TAIL_CHECKING_ENABLED 

(0x20), 

HEAP_FREE_CHECKING_ENABLED 

(0x40), 

HEAP_SKIP_VALIDATION_CHECKS 
(0x10000000)and 

HEAP_VALIDATE_PARAMETERS_ENA 
BLED (0x40000000). 

ForceFlags: 

HEAP_TAIL_CHECKING_ENABLED 

(0x20), 

HEAP_FRFE_CHECKING_ENABLED 


Calling NtQueryInformationProcess() with 
ProcessDebugPort as a ProcessInformationClass 
parameter will set Oxffffffff in the 
Processlnformation parameter if a process is being 
debugged [1]. Internally, such function queries for 
a non-zero state of EPROCESS->DebugPort. This 
function can be used to detect a debugger. 

Adopted Static Detection: 

NtQuerylnformationProcess is looked for in IAT. If 
found, the technique is considered as evidence 
detected. 


(0x40)and 

HEAP_VALIDATE_PARAMETERS_ENA 3.6. Debug Objects - 

BLED (0x40000000). ProcessDebugObjectHandle Class 


Adopted Static Detection: 

( 1 ) 

GetProcessHeap is looked for in IAT. If found, the 
technique is considered as an evidence detected. 

( 2 ) 

An instruction referencing PEB (fs:[0x30]) is 
looked for. If found, the first operand (X) is saved 
for future use: 

? X,? -> The substring “fs:[0x30]” is looked for in 
all the operands. If found, the first operand (X) is 
saved 

Then, later in the same function, any other 
instruction referencing the process default heap 


A debug object is created and a handle is associated 
to it when a debugging session begins [1]. 
NtQueryInformationProcess() can be called with 
ProcessDebugObjectHandle as a 
ProcessInformationClass parameter to query for the 
debug object handle. This can be used to detect the 
presence of a debugger. 

Adopted Static Detection: 

NtQuerylnformationProcess is looked for in IAT. If 
found, the technique is considered as an evidence 
detected. 

3.7. Debug Objects - ProcessDebugFlags Class 

NtQueryInformationProcess() can be called with 
ProcessDebugFlags as a ProcessInformationClass 
parameter to set the inverse of EPROCESS- 



>NoDebugInherit bit in Processlnformation 
parameter [1], So, FALSE is set when a debugger is 
present. This can be used to detect a debugger. 

Adopted Static Detection: 

NtQuerylnformationProcess is looked for in IAT. If 
found, the technique is considered as evidence 
detected. 

3.8. NtQuerySystemlnformation - 
SystemKernelDebuggerlnformation 

NtQuerySystemInformation() function of ntdll can 
be used with the undocumented 
SystemKernelDebuggerlnformation as a 
SystemlnformationClass parameter to detect the 
presence of a debugger [1], The result, that is 
stored in the buffer pointed by Systemlnformation 
parameter [23], has 2 bytes representing two flags, 
each one with 8 bits: KdDebuggerEnabled (least 
significant bye) and KdDebuggerNotPresent (most 
significant byte). KdDebuggerNotPresent is 
FALSE if a debugger is present. 

It is possible to obfuscate such a function call by 
retrieving KdDebuggerNotPresent directly from 
KUSER_SHARED_DATA, at offset 0x7ffe02d4 
for 2Gb user-space configurations. The value 
retrieved by the NtQuerySustemInformation() call 
does not come from this location. [2] 

This can be used to the detect the presence of a 
kernel-mode debugger [22], 

Adopted Static Detection: 

NtQuerySystemlnformation is looked for in IAT. If 
found, the technique is considered as evidence 
detected. 

3.9. OpenProcess - SeDebugPrivilege 

With SeDebugPrivilege privilege, a non-default 
privilege [5], a process can gain full control over 
the system process CSRSS.exe [1]. Additionally, 
such a privilege is passed to child processes. So, if 
a debugger acquires such a privielge, the debugged 
binary can have full control over CSRSS.exe also. 
[5] This technique has 2 steps: 

1. Enumerate processes to get the process ID 


of CSRSS.exe. This can be achieve through 
CreateToolhelp32 Snapshot()+ 
(Process32First())+Process32Next(). 
Another way could be using 
NtQuerySystemInformation() with 
SystemProcessInformation as a 
SystemlnformationClass parameter. 
Alternatively, Windows XP introduced the 
ntdll CsrGetProcessId() which makes such a 
task easier and can also be used. 

2. Open CSRSS.exe process with full access. 

If the operation succeeds, than it is an 
evidence of the presence of a debugger. 

This task can be achieved with 
OpenProcess() using 
PROCESS_ALL_ACCESS as a 
dwDesiredAccess parameter. 

OllyDbg and WinDbg acquires SeDebugPrivilege 
privilege. 

This technique might be used to indirectly detect 
the presence of some debuggers. 

Adopted Static Detection: 

The string “csrss.exe” is looked for in the binary in 
a case-insensitive way. If found, this technique is 
considered as evidence detected. 

3.10. Alternative Desktop 

Windows NT-based platforms supports multiple 
desktops, and it is possible to select a different 
active desktop, hiding the windows of the 
previously selected one with no obvious way to 
switch back to the old desktop [1], This can be 
done calling CreateDesktopO followed by 
SwitchDesktop(). The dwDesiresAccess parameter 
of CreateDesktopO can be: 
DESKTOP_CREATEWINDOW | 

desktop_writeobjects | 

DESKTOP_SWITCHDESKTOP. This technique 
can be used to make the debugging process harder 
for an analyst. 

Adopted Static Detection: 

CreateDesktopA/CreateDesktopW are looked for in 
IAT. If found, and SwitchDesktop is also present, 



the technique is considered as detected. 

3.11. Self-Debugging 


3.13. Hardware Breakpoints 


When an exception occurs, Windows passes to the 
exception handler a context structure which have, 
“Self-debugging is the act of running a copy of a among other information, the debug registers 
process, and attach to it as a debugger.” [1], Since content [1], If there is a debugger with hardware 
only one debugger can be attached to a process, breakpoints being used and it passes the exception 

such process could not be debugger by ordinary to the debuggee, then the debug registers can be 

means (there are bypasses). It is possible to execute analyzed looking for a debugger. This can be used 


this technique creating a copy of the process to be 
debugged (CreateProcessAQ with 
DEBUG_PROCESS as a dwCreationFlags 
parameter), and handling its debug events 
(WaitForDebugEvent() and 
ContinueDebugEvent()). This technique can be 
used to difficult a debugger to be attached to the 
process. 

Adopted Static Detection: 

CreateProcessA/CreateProcessW are looked for in 
IAT. If found, and both WaitForDebugEvent and 
ContinueDebugEvent are also present, the 
technique is considered as evidence detected. 


to detect the presence of a debugger. 

Adopted Static Detection: 

A MOV instruction (mov, movsx, movzx) copying 
the ESP register to the SEH (fs:[0x0]) is looked for: 

mov/movsx/movzx opl,esp -► Where “fs:[0x0]” is 
a substring of opl 

Then, later in the same function, another MOV 
instruction (mov, movsx, movzx) referencing the 
CONTEXT ([esp+Oxc]) is looked for in the source 
operand and the destination one (X) is saved for 
future use: 


3.12. RtlQueryProcessDebuglnformation 


mov/movsx/movzx X,op2 - 
a substring of op2 


where “[esp+Oxc]” is 


Then, later in the same function, instructions CMP 
(cmp, cmpxchg), MOV (mov, movsx, movzx) or 
OR, both with, in the source operand, having an 


RtlQueryProcessDebugInformation() is used to 
load some process information in DebugBuffer 
parameter including some heap information (heap 
flags is among them) [2] [3] [4], This call can be 
made with PDI_HEAPS | PDI_HEAP_BLOCKS as offset of a debug register relative to the saved X 
a DebuglnfoClassMask parameter. Internally, it (0x4, 0x8, OxC 0x10) is looked for: 
uses RtlQueryProcessHeapInformation(), and this mov/mosx/mozx opl,op2 
function can be used to develop a variation of this cmp/cmpxchg opl,op2 
technique. The following heap flags value indicates or opl,op2 

that a process is being debugged: GROWABLE | -> All op2 having “[X+0x4]”, “[X+0x8]”, 

TAIL_CHECKING_ENABLED | “[X+0xC]” or “[X+0xl0]” substrings. 

FREE_CHECKING_ENABLED | 

VALIDATE_PARAMETERS_ENABLED. This 


RET was used to consider the end of a function. 


technique can be used to detect the presence of a 

debugger. If this scenario happens, this anti-debugging 

technique is considered as detected. 


Adopted Static Detection: 


3.14. OutputDebugString 


RtlQueryProcessDebuglnformation and 

RtlQueryProcessHeapInformation are looked for in The kernel32 OutputDebugString() has different 
IAT. If some ot them are found, technique is behavior depending on the presence, or not, of 

considered as evidence detected. debugger [1], One of them is that kerne!32 



GetLastError() returns 0 if the debugger is present. Adopted Static Detection: 

This technique can be used to detect the presence 
of a debugger. (1) 

GetCurrentProcessId and 

CreateToolhelp32 Snapshot are looked for in I AT. If 
both are found, this technique is considered as 
evidence detected. 

( 2 ) 

3.15. Blocklnput GetCurrentProcessID and 

NtQuerySystemlnformation are looked for in IAT. 
BlockInput() function “Blocks keyboard and mouse If both are found, the technique is considered as 
input events from reaching applications.” [6]. This evidence detected, 
function can be used to difficult the access of an 
analyst to the debugger [1][5], (3) 

GetShellWindow, GetWindowThreadProcessId and 
NtQuerylnformationProcess are looked for in IAT. 
If both are found, this technique is considered as 
evidence detected. 

3.16. Parent Process 3.17. Device Names 

The parent process of an application executed by anDebuggers that uses kernel-mode drivers may use 
user will usually be “Explorer.exe”, and it can be named devices to communicate with them [1], So, 
considered as a debugger evidence when such a if an open attempt in such devices succeeds, it does 
characteristic does not happen [1], The following not necessarily means that a debugger is active, but 
functions can be used for this purpose: that it is present. The implementation can use 

• GetCurrentProcessId() + CreateFile() function with OPEN_EXISTING as a 

CreateToolhelp32Snapshot()+ dwCreationDisposition parameter. Some device 

(Process32First())+Process32Next(). names: 

• GetCurrentProcessId() + • SoftICE: WASICE, WASIWVID, WANTICE 

NtQuerySystemInformation() with • RegMon: WAFILEVXG, WAREGSYS 

SystemProcessInformation as a • FileMon: WAFILEVXG, WAFILEM 

SystemlnformationClass parameter. • WATRW 

• A simpler method: get Explorer.exe process . SoftICE extender: WAICEEXT 

ID (GetShellWindow() 

+GetWindowThreadProcessId()) and get thexhis technique might be used to detect the presence 
parent process ID of a debugger. The presence of a debugger does not 

(NtQuerylnformationProcessO with necessarily means that the debugger is active. 

ProcessBasicInformation as a 

ProcessInformationClass parameter). Adopted Static Detection: 


Adopted Static Detection: 


Blocklnput is looked for in IAT. If found, the 
technique is considered as evidence detected. 


Adopted Static Detection: 


OutputDebugStringA/OutputDebugStringW are 
looked for in IAT. If found, and GetLastError is 
also present, the technique is considered as 
detected. 


This technique might be used to detect the presence Device name strings (“WASICE”, “WASIWVID”, 
of a debugger. “WANTICE”, “WAFILEVXG”, “WAREGSYS”, 

“WAFILEM”, “WATRW”, “WAICEEXT”) are looked 



for in the binary itself in a case insensitive way. If 

found, this technique is considered as detected. Adopted Static Detection: 


3.18. OllyDbg - OutputDebugString 


SuspendThread and NtSuspendThread are looked 
for in IAT. If some of them found, this technique is 
considered as evidence detected. 


OllyDbg is a debugger that have a format string 
vulnerability with the kernel32 

OutputDebugString() function, leading to a crash or3.21. SoftICE - Interrupt 1 
an arbitrary code execution [1][5], The current final 
version (1.10) is still vulnerable. This can be used Normally, the DPL of interrupt 1 is set to 0, 


to break a debugging process with an affected 
version of OllyDbg. 

Adopted Static Detection: 

OutputDebugStringA/OutputDebugStringW are 
looked for in IAT. If found, this technique is 
considered as evidence detected. 

3.19. FindWindow 


meaning that a ring 3 attempt to execute int 1 
(“OxcdOl”) results in a CPU general protection 
fault (int “OxOd”) and in the end Windows raises an 
EXCEPTION_ACCESS_VIOLATION 
(0xc0000005) [1], 

SoftICE hooks IDT entry of interrupt 1 and sets the 
DPL to 3, allowing it to single-step from user-mode 
code. The problem is that SoftICE does not identify 
and handle differently the situations that caused 
such an int 1, and always execute the default 
interrupt 1 handler. 

FindWindow() function can be used to find opened So, a ring 3 attempt to execute int 1 results in the 
debuggers using both parameters, lpClassName andWindows raising EXCEPTION_SINGLE_STEP 
lpWindowName [1]. Some parameters that can be instead of EXCEPTION_ACCESS_VIOLATION 
used: (0x80000004). This characteristic can be used to 

• lpClassName: OllyDbg: “OLLYDBG”; detect if the SoftICE is running. 

WinDbg: “WinDbgFrameClass”; MSLRH: 

“TESTDBG”, “kkl”, “Eew57”, “Shadow”. Adopted Static Detection: 

• lpWindowName: MSLRH: “Import 

REConstructor vl.6 FINAL (C) 2001-2003 INTI instruction is looked for. Then, later in the 
MackT/uCF”. same function, a CMP instruction with 0x80000004 

in any of the operands is looked for: 

This can be used to detect the presence of a 
debugger. 


Adopted Static Detection: 

FindWindowA/FindWindowW are looked for in 
IAT. If found, this technique is considered as 
evidence detected. 


cmp operands -> where any of the operands are 
0x80000004 

RET was used to consider the end of a function. 


3.20. SuspendThread 

User-mode debuggers like OllyDbg and Turbo 
Debug can be disabled by calling kernel32 
SuspendThread() (or the ntdll NtSuspendThread()) 
in its threads [1][2], To find the threads, process 
enumeration and named window searching are two 
methods that can be used. 


If this scenario happens, this anti-debugging 
technique is considered as detected. 



3.22. SS register 


exception. [9] 

This behavior can be used to detect the presence of 
While single-stepping through trap flag, debuggers a debugger by defining a top-level exception filter 
typically try to clean such a flag when it is pushed through SetUnhandledExceptionFilter() and then 
in the stack [1][2][7], forcing an exception to occur. If the top-level 

When SS register is loaded (POP SS, for example), exception filter gets executed, then the process is 
the interrupts are disabled until the end of the next not being debugged, 
instruction to avoid invalid stack troubles in some 

cases [8], Adopted Static Detection: 

So, after the SS loading, the next instruction will be 

executed but the debugger will not break on it. SetUnhandledExceptionFilter is looked for in IAT. 

With the debugger unaware of the flags pushing If found, this technique is considered as evidence 

(PUSHFD, for example), the trap flags will not be detected. 

cleaned in the stack and its presence indicates a 

single-stepping thought trap flags debugging. 3.24. Guard Pages 

Adopted Static Detection: An attempt to access an address within a guard 

page (page marked with PAGE_GUARD) results in 
A POP instruction with SS register as operand, or a a STATUS_GUARD_PAGE_VIOLATION 
MOV instruction (mov, movsx, movzx) having SS (0x80000001) being raised by the system [1][10]. 
register as a destination operand are looked for: If a debugger is present, it might handle such an 

exception and allow the access. This behavior 

pop ss might be used to detect the presence of a debugger. 

An implementation of such a technique, as shown 

mov/movsx/movzx ss,? -► It does not matter what in [1], relies on writing 0xC3 (RET instruction) in a 
is the second operand memory area and marking this page with 

PAGE_GUARD. If the RET gets executed, the 

Then, the next instruction is analyzed to check if debugger is detected; otherwise, a crafted exception 
the next mnemonic starts with the string “pushf”. handler is executed meaning that the debugger was 

not detected. 

If this scenario happens, this anti-debugging 

technique is considered as detected. Adopted Static Detection: 

3.23. UnhandledExceptionFilter VirtualAlloc/VirtualAllocEx and 

VirtualProtect/VirtualProtectEx are looked for in 
When an exception is generated and there was no IAT. If found, this technique is considered as 
exception handlers to processes it, a default handler evidence detected, 
exists to do such a job [1][5][7]. As part of the 

default handler procedures, kernel32 3.25. Execution Timing 

UnhandledExceptionFilter() is called. In such a 

function, NtQueryInformationProcess() is called When a debugger is present, the time elapsed 
with ProcessDebugPort as a between subsequent instructions execution might 

ProcessInformationClass parameter to detect if the be higher than without it [1][2][7], The idea is to 
process that raised the exception is being debugged, measure time elapsed between some instructions 
If SetUnhandledExceptionFilter() was used and the execution and based on such a value, infer the 
process is not being debugged, the top-level presence of a debugger. Some methods can be used 

exception filter set by such a function will be to implement this technique (each method has its 

executed. Otherwise, if the process is being own characteristics): 

debugged, the debugger will be notified about the • RDTSC instruction (it is a popular anti- 



debugging technique [1] [2] [7] [11] but ThreadlnformationClass parameter has an 
there are some issues to be aware of [8] [11] undocumented value, ThreadHideFromDebugger 
[12] [13]) (Oxll), which prevents debugging events to be sent 

RDPMC instruction [2] [8] to the debugger [5]. This can be used to difficult the 

RDM SR instruction [2] [8] debugging. 

kernel32 GetTickCount() [14] 

winmm timeGetTime [1] [15] Adopted Static Detection: 

kernel32 GetLocalTime() [2] [16] 

kemel32 GetSystemTimeO [2] [17] NtSetlnformationThread is looked for in IAT. If 

kernel32 QueryPerformanceCounter() [2] ° und " “ s technic l ue is considered as evidence 
[7] [18], detected. 


Adopted Static Detection: 3 ' 28 ' NtSetDebugFilterState 


GetTickCount, timeGetTime, GetLocalTime, 
GetSystemTime and QueryPerformanceCounter are 
looked for in IAT. If some of them are found, this 
technique is considered as evidence detected. 

3.26. Software Breakpoint Detection 

Software breakpoint is a single-byte instruction 
(OxCC - INT 3) that stops the execution of the 
debugged process and passes control to the 
debugger [5]. The original byte is saved by the 
debugger before setting the breakpoint, this way the 
original instruction can be executed in the correct 
time. [19] 

Code areas in memory are scanned for OxCC byte 
that was not set by the code itself. To make such a 
check not so obvious, it is possible to use some 
operation in the compared by, such as [5]: 

if(byte XOR 0x55 == 0x99) then breakpoint found 

Note that OxCC XOR 0x55 = 0x99. 

Adopted Static Detection: 

A CMP instruction (cmp, cmpxchg) with OxCC in 
any of its operands is looked for. If found, this anti¬ 
debugging technique is considered as detected. 

3.27. Thread Hiding 


The ntdll DbgSetDebugFilterState (or ntdll 
NtSetDebugFilterState) call succeeds in the 
presence of some debuggers [2], This is a side- 
effect of the debugger's behaviour: the process 
SeDebugPrivilege privilege. SeDebugPrivilege is 
not a default privilege [5], so this technique might 
be used to indirectly detect the presence of some 
debuggers. 

Adopted Static Detection: 

DbgSetDebugFilterState and 
NtSetDebugFilterState are looked for in IAT. If 
some of them are found, this technique is 
considered as detected. 

3.29. Instruction Counting 

An exception handler is registered to deal with the 
EXCEPTION_SINGLE_STEP (0x80000004) 
exception [1]. 

Then, some hardware breakpoints are set in specific 
instructions. Debug registers cannot be accessed 
directly in user-mode [32], so a context structure is 
needed and the following procedures can be used to 
get it: 

• Calling kernel32 GetThreadContext(). 

• Forcing an exception to occur and handling 
it, because the context structure is passed to 
the exception handler. This is more stealth 
than the previous procedure. 


According to MSDN [20] [21], ntdll 
NtSetInformationThread() sets the priority of a 
thread [1][5][7], However, its 


As instructions with hardware breakpoints are 
being reached, the previously registered exception 
handler is supposed to deal with the raised 




exceptions. Such handler will simply count how WaitForSingleObject functions are looked for in 
many times it was reached and then can change the IAT. If some of them are found, this technique is 
EIP to point to a new instruction and resume the considered as evidence detected, 
execution. 

Some debuggers do not deal correctly with 3.32. Hook Detection 

hardware breakpoints that were set by them, and 

some of the raised EXCEPTION_SINGLE_STEP Some hook techniques relies on overwriting the 
might not be handled by the previously set first instruction of the hooked function by a JMP 

exception handler. instruction pointing to another place. [48] 

After all hardware breakpoints got reached and its 

exception handlers finished, the total counter used Regarding Microsoft Detours, some characteristics 
by them should have the number of hardware exist that can be used as a signature, such as 
breakpoints initially set. If the value was different, .detours section and the presence of detoured.dll. 
it indicates the presence of a debugger. [49] [50] 

3.30. Header Entrypoint Detecting the presence of a hook might detect some 

binary analysis procedures. 

File sections that do not include the attribute 

IMAGE_SCN_MEM_WRITE (write) is read-only Adopted Static Detection: 
by default to a remote debugger [1]. 

Additionally, there is no section that describes the (1) 

PE header, it will be also considered as read-only; 

there is an exception when the PE- A CMP instruction with 0xE9 in some of its 

> Section Alignment is less than 4kb, which causes operands is looked for. If found this technique is 
it to be marked internally as both writable and considered as evidence detected, 
executable [1]. 

Being so, if the debugger does detect such situation (2) 
and does not set a write privilege in such a section, 

the debugger might allow the application to run The string “.detour” is looked for in the binary with 
freely. the exception of its sections. The string 

“detoured.dll” is also looked for in the binary, but 
Adopted Static Detection: with the exception of the imports. If some of them 

were found, the technique is considered as 
The entrypoint section is analyzed to check if it has detected, 
the IMAGE_SCN_MEM_WRITE attribute. If it 

does not have, then this technique is considered as Section 3.33. DbgBreakPoint Overwrite 
detected. 

When a debugger attaches to a process, an 

3.31. Self-Execution exception is raised by DbgBreakPoint() function in 

NTDLL (called at attach time) [2], Handling such 
This technique relies on a process to create another an exception the debugger gains control of the 
process of itself [1]. This way, the second process debugee. 
will not be debugged. Usually this trick is used 

with a mutex to prevent many copies of the process By marking the page(s) of DbgBreakPoint() as 
to be in execution. EXECUTE_READWRITE and overwriting it with, 

for example, a RET instruction, when a debugger 
Adopted Static Detection: attaches to the process the thread will exit 

immediately, thus, not breaking in. 

CreateProcessA/CreateProcessW, CreateMutex and 



4. Obfuscation and Anti-Disassembly 
Techniques 

Both obfuscation and anti-disassembly techniques 
relies on a disassembly. Being so, they were put 
together in the same section. 

Obfuscation is a kind of technique to make the 
disassembly result harder to be analyzed by a 
professional. 

Anti-disassembly is a kind of technique to 
compromise disassemblers and/or the 
disassembling process. 

Section 4.1 discusses some disassembly concepts. 
Section 4.2 and Section 4.3 describes, respectively, 
some obfuscation and anti-disassembly techniques. 

4.1. Disassembly Concepts 

It is possible to disassemble a binary with a static 
and a dynamic approach [39], The former relies on 
executing the program and tracking instruction as 
they are being executed. The latter relies on 
analyzing the program bytes and finding 
instructions without executing it. 

Static disassembling can be categorized in two 
main classes: linear sweep and recursive traversal. 
Linear sweep approach starts from a given byte (for 
example, the first byte of the entry point) and from 
this point on analyzes byte after byte until a 
predefined end (for example, the end of the PE 
section). The main drawback of this approach is 
that data placed in the middle of code instructions 
may generate some noise, because they will be 
interpreted as code. An example of disassembler 
that uses linear sweep approach is objdump [26]. 
Recursive traversal is an approach that follows the 
program control flow instead of simply 
disassembling each byte. It is not vulnerable to the 
simple fact of data existing in the middle of code 
instructions, but it has another main drawback: it is 
not always possible to statically predict the exact 
program control flow. It may result in some parts 
not being disassembled and also the generation of 
some noise. The unreachable areas can be 
submitted to a linear sweep processing, and such a 
variation is called speculative disassembly. An 
example of disassembler that uses recursive 
traversal approach is IDA [40]. 


Anti-disassembly techniques are discussed in 
section 4.2 and obfuscation techniques in section 
4.3. 

4.2. Anti-Disassembly Techniques 

Some anti-disassembly techniques are described in 
the next sections. 

Techniques currently covered by detection plugins 
will have an additional information: the algorithm 
used to detect such a technique. 

4.2.1. Garbage Bytes 

This technique relies on adding additional bytes 
that will never be executed in run-time. [5] [38] 
This may break both linear sweep and recursive 
traversal approaches. 

A liner sweep approach could interpret such bytes 
as being code-related bytes, breaking the 
alignment. As a result, such garbage bytes could be 
joined with valid bytes from next instructions 
generating wrong instructions instead of the correct 
ones. For example: 

jmp .destination 
db 0x6a ; garbage byte technique 

.destination: 

pop eax 

Such example generates the following disassembly 
by a objdump: 

eb 01 jmp 0x401003 

6a 58 push 0x58 

Recursive traversal algorithms might also be 
compromised through garbage bytes if a situation 
in which the same set of bytes with more than one 
interpretation could be forced. In this case, the lack 
of alignment due to the interpretation of the 
garbage bytes as a valid code bytes might lead the 
disassembler to produce a wrong disassembly. For 
example, a Fake Conditional Jump implementation 
could be used for that: 

mov eax,eax 
jz .destination 



db 0x6a; garbage byte technique 
.destination: 

; rest of the code 
pop eax 

Such example produces the following IDA output: 


Resulting objdump output: 

eb 01 jmp 0x401003 

6a 58 push 0x58 


# IDA output 
Adopted Static Detection: 


Another example of implementation is the 
Instruction Substitution that uses a Push followed 
by RET to replace a conventional JMP. 


( 1 ) 

A PUSH instruction immediately followed by a 
RET is looked for. If found, the technique is 
considered as evidence detected. 

( 2 ) 


It is also possible to use this technique to 
compromise recursive traversal algorithms by using 
indirection. An indirect jump, for example, is an 
approach that can be used for such a purpose. [39] 
[41] The previous example was modified to use an 
indirect jump: 

push DWORD .destination 


A XOR instruction with two equal operands is 
looked for. If found and is immediately followed by 
a JNZ instruction, the technique is considered as 
evidence detected. The same happens for STC 
instruction immediately followed by JNC or JAE 
and for CLC instruction immediately followed by 
JC or JB. 


jmp DWORD [esp] 

db 0x6a ; garbage byte technique 

.destination: 

pop eax 

##### IDA output ######## 


Adopted Static Detection: 


4.2.2. Program Control Flow Change 

A PUSH instruction immediately followed by a 

This technique relies on unconditionally forcing a RET is looked for. If found, the technique is 
program control flow change to occur, leaving an considered as evidence detected, 
area with other anti-disassemble technique(s) 

unreachable in run-time. Disassemblers using linear4.2.3. Fake Conditional Jumps 
sweep approach will disassemble such an area and 

the resulting assembly code may be compromised. This technique, based on [5] and [38], relies on 

creating conditional jumps which conditions are 
An unconditional JMP is an example that can be always the same. For example: 
used to implement this technique. [38] For 
example, the following JMP instruction jumps an (1) 
unreachable area populated with Garbage Byte 
anti-disassembly technique, avoiding its execution, xor eax,eax 
But objdump will disassemble such an area and the jz .destinationl; always true 
resulting output is compromised: 


jmp .destination 
db 0x6a; garbage byte technique 
.destination: 

; rest of the code 
pop eax 


( 2 ) 

xor eax, eax 

jnz .destination ; always false 



function. After the CALL and before the next 

In the first example, the JZ instructions will be executed instruction, other anti-disassembly 
always true independently of the EAX content, the techniques, such as Garbage Bytes, can be used, 
instructions before XOR and the instructions after Linear sweep is also affected because they do not 
JZ. The same happens for the second example, but interpret instructions and may also disassemble the 
the JNZ instruction will be always false. next instruction after the call, getting vulnerable to 

Recursive traversal approach may disassemble other anti-disassembly techniques such as Garbage 

areas that will never be executed, and such Bytes. 

unreachable areas can be populated with other anti- The following example, which also employs 
disassembly techniques, such as Garbage Bytes, Garbage Bytes technique, may break for both, 
that creates two different interpretations for the recursive traversal and linear sweep approaches: 
same set of bytes. 

Each disassembler has its own way to handle such call .function 

a conflict, but most of them, trust its first db 0x6a ; garbage byte 

interpretation [38]; IDA seems to be an example of .correct_return: 
this, because it first disassembles the false branch ; rest of the code 

[38]. pop eax 


The following approaches are examples that can be 
used to implement this technique: 

• xor x,x (XOR with two equal operands) 

° True branch: JZ 
° False branch: JNZ 

• STC instruction 


.function: 

push 
ret 


DWORD ,correct_return 


The following output is produced by objdump: 


° True branch: JC or JB 
° False branch: JNC or JAE 
CLC instruction 
° True branch: JNC or JAE 
° False branch: JC or JB 


401000 

401005 

401007 

40100c 


e8 02 00 00 00 
6a 58 

68 06 10 40 00 
c3 


call 0x401007 
push 0x58 
push 0x401006 
ret 


Adopted Static Detection: 


The following output is produced by IDA: 


A XOR instruction with two equal operands is 
looked for. If it is immediately followed by JNZ 
instruction, the technique is considered as detected. 
The same happens for STC instruction immediately 
followed by JNC or JAE and for CLC instruction 
immediately followed by JC or JB. 

4.2.4. Call Trick 


# IDA output 

4.2.5. Flow Redirection to the Middle of an 
Instruction 

This technique relies on redirecting the program 
flow to the middle of an instruction. [38] This 
might compromise both linear sweep and recursive 
traversal algorithms. 


This technique relies on changing the default 
function's return address.[39] [41] In conjunction 
with other techniques such as Garbage Bytes, this 
trick may break all kind of disassemblers. 
Recursive traversal disassemblers may disassemble 
the next instruction after the CALL, but the correct 
next instruction was actually changed by the called 


An implementation example could be hiding an 
instruction in the middle of another. So, the 
disassembler would show an instruction that is not 
executed in run-time instead of the correct 
instruction that resides in the middle of its bytes. 
Linear sweep approaches could be bypassed 
because the instruction aligned to the rest of the 




bytes are the wrong one. Recursive traversal 
algorithms could be affected by making the same 
set of bytes to have more than one interpretation; 
this can be achieved, for example, by using the 
Fake Conditional Jump technique. 

The following example illustrates such a scenario 
with a code that affects both, linear sweep and 
recursive traversal approaches: 

; Fake Conditional Jump 
xor eax,eax 

jz +4 ; jump to the ret 

; 0xc3 = ret 

mov eax,0xc3abcdef 


; rest of the code 
xor eax,eax 
pop eax 
mov eax,esp 
push ecx 

Output of objdump: 

66 b8 eb 05 mov ax,0x5eb 

31 cO xor eax, eax 

74 f9 je 0x401001 

e8 31 cO 58 89 call 0x8998d03e 

eO 51 loopne 0x401060 

Output of IDA: 


In such an example, the RET instruction does not # IDA output 
directly appear in the disassembly outputs, but is 

executed in run-time, as shown in the objdump and This technique could also be used to make 
IDA outputs below. recursive traversal algorithms to generate two 

different interpretations for the same set of bytes 

Output of objdump: without using conditional jumps: jumping into 

itself [38], Additionally, because it breaks the 

31 cO xor eax,eax alignment, linear sweep algorithms may also be 

74 04 je 0x401008 affected. The following example, based on [38], 

b8 ef cd ab c3 mov eax,0xc3abcdef illustrates such a scenario: 


Output of IDA: ; All bytes of the example: 

; Oxeb Oxff OxcO 0x48 

# IDA output 

; jmp -1 = Oxeb Oxff 

Another implementation example could be using ; jumps to itself: Oxff 

this anti-disassembly technique to break the jmp -1 

alignment and generate a set of wrong instruction 

instead of simply hiding one in the middle of ; Oxff OxcO = inc eax 

another. The following example, that is based on db OxcO 

[38], does this: 

; 0x48 = dec eax 

mov ax,0x05eb db 0x48 

xor eax,eax 

Output of IDA: 

; jump to “jmp 5” (Oxeb 0xe5) 

; last bytes of mov instruction is Oxeb 0xe5 # IDA output 
; such “jmp 5” redirects the flow to the rest 
; of the code Output of objdump: 

jz -6; 


db 0xe8 ; garbage byte 


eb ff 

cO 

48 


jmp 0x401001 
byte OxcO 
dec eax 



future use: 


A PUSH instruction immediately followed by a 
RET is looked for. If found, the technique is 
considered as evidence detected. 


pop X 

Then, the next instruction is compared against 
AND, OR and XOR with the destination operand 
being the saved one (X) and the other one being an 
immediate: 


( 2 ) 


and/or/xor X,immediate 


A XOR instruction with two equal operands is If this scenario happens, the technique is 
looked for. If found and is immediately followed by considered as detected, 
a JNZ instruction, the technique is considered as 

evidence detected. The same happens for STC 4.3.2. NOP Sequence 
instruction immediately followed by JNC or JAE 

and for CLC instruction immediately followed by This type of dead-code insertion relies on adding a 
JC or JB. sequence of NOP instructions in the middle of the 

code [25]. This can make the disassembly analysis 
4.3. Obfuscation Techniques harder by reducing the legibility of the code and 

bypassing some signature-based algorithms. 

Some obfuscation techniques are described in the 

next sections. Adopted Static Detection: 

Techniques currently covered by detection plugins 

will have an additional information: the algorithm A sequence of 5 NOPs is looked for in the same 
used to detect such a technique. function. RET was used to consider the end of a 

function. 

4.3.1. Push Pop Math If found, this technique is considered as detected. 


This technique can be used to obfuscate a value and4.3.3. Instruction Substitution 


relies in three steps [24]: 

• Push a known immediate 

• Pop such an immediate into a register 

• Do some math on the register 

At the end, the register will have the desired value, 
but such a value does not explicitly appear in the 
code itself. 

Adopted Static Detection: 

A PUSH instruction with an immediate operand is 
looked for: 

push immediate 

If found, the next instruction is compared against a 
POP: if it is true, the destination (X) is saved for 


This technique relies on changing a instruction, or a 
set of them, by equivalent ones. [25] [45] It can be 
used to make the analysis process by a professional 
harder and also to bypass signatures. Some 
examples are: 

• “xor eax,eax -> jz” to replace a JMP 

P. For example, “jmp .destination” can be 
replaced by “xor eax,eax -► jz 
.destination” 

• “push - pop” to replace a MOV 

p For example, “mov eax,0xl” can be 
replaced by “push Oxl pop eax” 

• “sub” to replace a XOR 

° For example, “xor eax,eax” can be 
replaced by “sub eax,eax” 

Another example, that will be discussed in more 



details, is to replace a JMP by “push - ret”. 

According to [8], RET “transfers program control 
to a return address located on the top of the stack” 
and, additionally, it pops such an address to EIR 
So, if the stack gets manipulates to put in its top the 
desired address to transfer the program control flow 
to, RET and its variations, such as RETN and 
RETF, can be used as an obfuscated JMR 
The most known way to implement such a 
technique is the Push Ret: the address to redirect 
the flow to is pushed and then RET is called 
effectively changing the flow: 

push .destination 
ret 

Although Push Ret is the most known approach, 
there are other variations, for example: 

mov [esp],DWORD .destination 
ret 

RET is often used to return from a procedure. 

Being so, if the alternative jump variation seems 
like a given calling convention function prolog, it 
would be more stealth and more difficult to 
automatically detect. For example: 


push 

.destination 

push 

ebp 

mov 

ebp,esp 

leave 


ret 



Adopted Static Detection: 

PUSH instruction is looked for. If found and the 
next instruction is a RET, then the technique is 
considered as detected. 


Shuffle the instructions and make them to 
be executed in the correct order by using 
program control flow changes. This can be 
achieved, for example, by using 
unconditional jumps and some Instruction 
Substitutions of it such as “xor eax,eax - jz” 
being used instead of a JMP instruction. 
Choose and reorder set of instructions that 
does not interfere in each other results. So, 
such a shuffling process will change the 
order of instructions in the binary and at the 
time does not change the program results 

As an example, the following code is considered as 
the binary before the obfuscation process: 

xor eax,eax 
inc eax 
push ebx 


The following code is an example of the original 
binary obfuscated with the program control flow 
changes approach: 

jmp .first 
.second: 

push ebx 

jmp .continuation 

.first: 

xor eax,eax 

inc eax 
jmp .second 
.continuation: 


The following code is an example of the original 
binary obfuscated with the reordering approach: 

push ebx 


4.3.4. Code Transposition ; inc depends on xor 

; so such instruction order was not changed 
This technique relies on shuffling instructions so xor eax,eax 

that the order they appear in the binary gets inc eax 

different from the order they were executed [25] 

[45], 

The following two methods can be used for such a 
purpose: 



4.3.5. Register Reassignment 


.destination: 


This technique relies on changing the registers used 

by a program or part of it [25][45]. Instead of using a simple JMP instruction, any 


For example, the following code shows a program 
before the obfuscation: 

xor eax,eax 
inc ebx 

After a fictitious obfuscation which exchanges 
EAX by EBX and vice-versa, the following code 
will be generated: 

xor ebx, ebx 
inc eax 

Although this technique does not make an analysis 
much more complicated, it can be used to bypass 
signatures. 


other technique that can be used to redirect the 
program control flow, such as Fake Conditional 
Jump and Code Substitution, could be used. For 
example: 

(1) Fake Conditional Jump example 
xor eax,eax 

jnz ,fake_code 
jmp .destination 
.fake_code: 

push 0x12345678 ; fake code 
inc eax; fake code 
mov esp,eax; fake code 
; more fake code here 
.destination: 


4.3.6. Code Integration 

This technique relies on disassembling a target 
program file, inserting the code to be obfuscated 
inside it [45] [46], In order to do that, the target 
program needs to be fixed. This way, the code to 
obfuscated is hidden in the middle of the other 
program. 


be 


(2) Code Substitution example 
push .destination 
ret 

push 0x12345678;fake code 
inc eax; fake code 
mov esp,eax; fake code 
; more fake code here 
.destination: 


4.3.7. Fake Code Insertion 


Adopted Static Detection: 


This is a variation of Garbage Bytes anti¬ 
disassembly technique. The idea is to insert 
instructions that will never be executed [38], 
making them to appear in the generated 
disassembly. This can, for example, confuse the 
professional that is analyzing the disassembly with 
lots of fake code and bypass signature-based 
algorithms. 

The implementation is exactly the same as Garbage 
Bytes technique, but instead of adding garbage 
bytes, valid instructions are added. 

jmp .destination 
push 0x12345678 ; fake code 
inc eax; fake code 
mov esp,eax; fake code 
; more fake code here 


( 1 ) 

A PUSH instruction immediately followed by a 
RET is looked for. If found, the technique is 
considered as evidence detected. 

( 2 ) 

A XOR instruction with two equal operands is 
looked for. If found and is immediately followed by 
a JNZ instruction, the technique is considered as 
evidence detected. The same happens for STC 
instruction immediately followed by JNC or JAE 
and for CLC instruction immediately followed by 
JC or JB. 



4.3.8. PEB->Ldr Address Resolving 

PEB is a structure that contains process 
information. Among its fields, there is the Ldr, 
which points to a structure that contains 
information about the loaded modules for the 
process. [34] 

It is possible to retrieve the PEB (fs:[0x30]) and 
access its Ldr field (0x0c). So, the loaded modules 
can be accessed and function addresses resolved. 
[34] [35] [36] 

Adopted Static Detection: 

A MOV instruction (mov, movsx, movzx) copying 
PEB address (fs:[0x30]) somewhere (X) is looked 
for and X is saved for future use: 


has been found. From this point on, the 
IMAGE_DATA_DIRECTORY entry of the DLL 
can be found using the 0x78 offset to get the RVA 
to the export directory, which, together with the 
previously found handle, results in the Export 
Directory Table address. 

Adopted Static Detection: 

( 1 ) 

A MOV instruction (mov, movsx, movzx) copying 
SEH address (fs:[0x0]) somewhere (X) is looked 
for and X is saved for future use: 

mov/movsx/movzx X,op2 -> Where “fs:[0x0]” is 
inside op2 


mov/movsx/movzx X,op2 -► Where “fs:[0x30]” is 
inside op2 

Then, later in the same function, a MOV (mov, 
movsx, movzx) or a CMP (cmp, cmpxchg) 
instructions referencing the Ldr ([X+0x0c] in some 
of the operands) are looked for: 

mov/movsx/movzx/cmp/cmpxchg opl,op2 4' 
where [X+OxC] is a substring of opl or op2 

RET was used to consider the end of a function. 


Then, later in the same function, a MOV (mov, 
movsx, movzx) instruction referencing the PEB (fs: 
[0x30]) in the second operand is looked for and, if 
found, the algorithm is reseted. 

Continuing with the next lines, a MOV instruction 
(mov, movsx, movzx) referencing the exception 
handler ([X+0x4]) in the second operand is looked 
for and, if found, the first operand (Y) is saved for 
future used: 

mov/movsx/movzx Y,op2 -> where “[X+0x4]” is a 
substring of op2 


If this scenario happens, the technique is 
considered as detected. 

4.3.9. Stealth Import of the Windows API 

Regardless of the import table, ntdll.dll and 
kernel32.dll are automatically mapped into process 
address space [37], It means that it is possible to 
access them even in an executable with no imports. 
Such DLLs can be access through SEH, because its 
first record normally points to either ntdll.dll or 
kernel32.dll. 

To get the DLL address, the SEH could be walked 
until the first element, which 0x4 offset is the 
handler field. Then, it is possible to scan the 
memory looking for 'MZ' and, once found, check if 
it is in the correct place through 0x3C offset that is 
supposed to be “PE\0\0”: a handle to the module 


Later in the same function, a CMP instruction 
(cmp, cmpxchg) referencing Y in the first operand 
is looked for: 

cmp/cmpxchg opl,? -> Where Y is a substring of 
opl 

Later in the same function, a MOV instruction 
(mov, movsx, movzx) with the “PE\0\0” offset 
relative to Y ([Y+0x3c]) in the second operand is 
looked for: 

mov/movsx/movzx ?,op2 -> Where [Y+0x3c] is a 
substring of op2 

Later in the same function, instructions AND, OR, 
XOR, ADD or SUB CMP referencing the 
IMAGE_DATA_DIRECTORY offset (0x78) in 



some of the operands is looked for 

and/or/xor/add/sub ? -► Where “0x78” is a 
substring in any of the operands 

RET was used to consider the end of a function. 

If this scenario happens, the technique is 
considered as detected. 


[8] [27] [28] [30] 

• SGDT: Stores the Global Descriptor Table 
Register (GDTR) content. [8] [27] [28] [30] 

• STR: Stores the segment selector from the 
Task Register (TR). [8] [27] [28] [30] 

• SMSW: Stores the machine status word into 
the destination operand . [8] [30] [42] 

Adopted Static Detection: 


(2) Instructions SIDT, SLDT, SGDT and STR are 

looked for. If some of them are found, this 

If the IAT is empty, this technique is considered as technique is considered as detected, 
evidence detected. 


4.3.10. Function Call Obfuscation 

LoadLibrary and GetProcAddress functions can be 
used to call any other. By only importing these two 
functions is possible to obfuscate function calls. 

Adopted Static Detection: 


If 


5.2. VMWare - IN Instruction 

I/O ports can be accessed through the privileged 
instructions IN and OUT: in normal cases [31] an 
attempt to run such instructions in user-mode will 
generate an exception. [28] [31] 

VMWare [43] uses IN instruction in a special port 
(VX), that exists only inside its virtual machines, as 
an interface between virtual machines and 
VMWare software itself. So, such operation will 
not generate an exception if executed in user-mode 
inside a VMWare virtual machine. [28] [31] 

This can be used to detect if an application is 
running inside a VMWare virtual machine. 


LoadLibrary A/LoadLibraryW/LoadLibraryExA/Lo 
adLibraryExW and GetProcAddress are both found 
in IAT, this technique is considered as detected. 

5. Anti-Virtual Machine 

Some anti-virtual machine techniques are described 
in the next sections. 

Techniques currently covered by detection plugins technique is considered as detected, 
will have an additional information: the algorithm 

used to detect such a technique. 5.3. VirtualPC - Invalid Instruction 


Adopted Static Detection: 


IN instruction is looked for. If it is found, this 


5.1. CPU Instructions Results Comparison When an invalid instruction is executed, an 

exception is raised and it can be handled by the 
Some CPU instructions, due to their specific software using try/catch mechanism [31]. 
nature, have characteristic results when executed VirtualPC [44] relies on invalid instructions to 
inside virtual machine solutions that can be used to interface between virtual machines and VirtualPC 
infer its presence. [28] software itself. An example is the invalid 

The following instructions are examples that can bei nstruct i on «0x0F 0x3F 0x07 OxOB”, which does 
used for such a purpose: not generate an exception inside a VirtualPC virtual 

• SIDT: Stores the Interrupt Descriptor Table machine. 

Register (IDTR) content. [8] [27], [28] [29] This can be used to detect if an application is 
[30] running inside a VirtualPC virtual machine. 

• SLDT: Stores the segment selector from the 
Local Descriptor Table Register (LDTR). 



Adopted Static Detection: 

Starting at a byte that were not recognize as valid 
by the disassembler, the following four byte 
sequence are looked for: 

OxOF 0x3F 0x07 OxOB 


This pattern generated no false-positives when 
tested against the 883 executables and correctly 
detected SSEXY encryption in all the 9 cases. 
SSEXY was released in May/2012 and in more or 
less one month later a detection plugin was 
finished, tested and running in the Dissect || PE 
system. 


If this scenario happens, the technique is 
considered as detected. 

6. New Techniques 

The new techniques implemented by this work are 
described by the next sections. 

6.1. Dynamic Approach 


Adopted Static Detection: 

The following pattern is looked for in the binary: 

66 OF 70 ?? ?? 66 OF DB ?? ?? ?? ?? ?? 66 
OF DB ?? ?? ?? ?? ?? 66 OF EF 

If found, the technique is considered as detected. 

6.3. Flame Detection 


The static techniques in the previous section, which 

relied on function calls or function calls with The ^ ame ma ^ ware mac ^ e news due to its rich 


specific parameters, are not reliably detected using 
only the static approach. 

Being so, a dynamic approach was develop that 
puts a software breakpoint in the target functions. 
When such functions are reached, it is possible to 
more reliably detect the call and extract the 
parameters. 


capabilities and to the fact that it remained 
undetected for long time. Many researchers quickly 
noted the existence of embedded scripting language 
within the malware and pointed this as a new 
enhancement for malwares. We wrote a detection 
script to inspect all our samples for the presence of 
embedded scripting language, such as Lua [47], 


6.2. SSEXY Detection 


7. Resources 


SSEXY [33] is a tool developed by Jurriaan 
Bremer that, given a binary, obfuscates it 
converting many “conventional” assembly 
instructions to an SSE-based version. In this work, 
it was considered as an obfuscation technique. 
There were some troubles running SSEXY in the 
883 executables used to test all plugins and 
techniques in this work because such a tool is still 
in an early development stage. So, it was developed 
some simple binaries for the specific purpose of 
testing the SSE obfuscation provided by SSEXY. 

At the end, together with the two demo binaries 
distributed with SSEXY, there were 9 cases to 
study the SSEXY obfuscation. The following 
pattern was identified in all the 9 cases: 

66 OF 70 ?? ?? 66 OF DB ?? ?? ?? ?? ?? 66 
OF DB ?? ?? ?? ?? ?? 66 OF EF 


The most updated version of this document can be 
found at: http://research.dissect.pe . 

Additionally, examples for each of the attacking 
techniques discussed in this paper are available at: 
https ://github. com/rrbranco/blackhat2 012. 

8. Conclusions and Future Directions 

This research provides a guidance on protecting 
techniques used by malware, more specifically the 
anti-debugging, anti-disassembly, obfuscation and 
anti-VM ones. It also extrapolates the current 
standards in malware analysis providing the results 
against millions of samples. 

We created examples for each of the techniques 
discussed in this paper, facilitating the development 
of the detection codes. Additionally, such codes are 
publicly available. 

For validation purposes, this work explains how the 










detections are being executed. [10] MSDN - Creating Guard Pages - 

http://msdn.microsoft.com/en- 

The research results can be expanded and hopefully us/library/windows/desktop/aa366549%28v=vs.85 
we will publicly release more information, such as: %29.aspx (Last Access: 25/June/2012) 

• More anti-reverse engineering techniques [11] Josh_Jackson - An Anti-Reverse Engineering 

• More statistics with more analyzed samples Guide - 

http://www.codeproject.com/Articles/30815/An- 
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